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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
01/07/2008 has been entered. 



Response to Amendment 

2. The amendment filed on 01/07/2008 has been entered. 

The amendment filed 01/07/2008 is objected to under 35 U.S.C. 132(a) because 
it introduces new matter into the disclosure. 35 U.S.C. 132(a) states that no 
amendment shall introduce new matter into the disclosure of the invention. The added 
material which is not supported by the original disclosure is as follows: Changing "real 
object" to "operating unit" is not supported by applicants originally filed specification. 
Refer to the discussion in the arguments section below concerning 112 first paragraph 
and the below 35 USC 112 first paragraph rejection. 

Applicant is required to cancel the new matter in the reply to this Office Action. 
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Response to Arguments 

3. Applicant's arguments filed 01/07/2008 have been fully considered but they are 
not persuasive for the following reasons. 
1 1 2 first paragraph rejection 

On page 1 1 applicant proffers that the amendments to independent claims 1 , 5, 
and 16 overcomes the 35 USC 1 12 first paragraph rejection and the amendment to the 
specification overcomes the new matter objection both of which were set forth in the 
office action mailed on 10/05/2007, however, the "operating unit" language added to the 
specification and these claims corresponds to "operating unit 1090" of applicants 
specification which does not generate position and orientation of a plurality of positions 
in real space "for setting a constraining shape by using a shape generated based on the 
inputted three-dimensional position information in case of the input of the constraining 
shape" (claims 1 , 5, and 19) and for "obtaining a constraining shape by using a shape 
generated based on the obtained three-dimensional position information" (claim 10). 
Note applicant also added "operating unit moved by a user in the real space" to claim 
10 in the 01/30/2007 amendment. Applicants specification in paragraphs 30, 31, 32, 44, 
69, 82, 83, 84, and 85 discuss operation unit 1090 as controlling the virtual object and in 
paragraphs 28, 30, 42, and 50 discuss stylus 1060 as setting the position and 
orientation of a plurality of positions in real space used "for setting a constraining shape 
by using a shape generated based on the inputted three-dimensional position 
information in case of the input of the constraining shape" (claims 1, 5, and 16) and 
used for "obtaining a constraining shape by using a shape generated based on the 
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obtained three-dimensional position information" (claim 10). The specification does not 
convey the operating unit 1090 is moved to measure the three-dimensional position 
information used "for setting a constraining shape by using a shape generated based on 
the inputted three-dimensional position information in case of the input of the 
constraining shape" (claims 1, 5, and 19) and used for "obtaining a constraining shape 
by using a shape generated based on the obtained three-dimensional position 
information" (claim 10). Thus, applicants originally filed specification does not convey 
that applicant had possession of the amendments to claims 1, 5, 10, and 16. 
1 03 rejection 

Argument 1 : 

On page 13 and first paragraph on page 14 applicant argues the obtaining step 
of claim 10: (1 ) obtaining three-dimensional position information of a plurality of 
positions designated by an operating unit moved by a user in the real space, the 
operating unit being capable of measuring the position and orientation. Applicant on 
page 13 last sentence argues "If the shapes of the real objects are known in advance, 
there would be no need to obtain three-dimensional position information in order to 
obtain a constraining shape. Further, nothing in Kitamura suggests that the 6 DOF 
tracker is capable of measuring position and orientation. " Applicant then on page 14 
argues "As discussed above, Kitamura does not teach the obtaining step of Claim 10 
because if fails to teach or suggest obtaining three-dimensional position information of a 
plurality of positions that are used to obtain a constraining shape, or an operating unit 
capable of measuring position and orientation.", however, both of these arguments are 



Application/Control Number: 10/615,941 Page 5 

Art Unit: 2628 

not persuasive because the shape of the real objects are known in advance to 
manipulating a virtual object, see section 5 first paragraph, which requires an input of 
the three-dimensional positions. Additionally section 2 at lines 7-14 states "To bring an 
object that already exist in the real world into a computer-generated virtual world, it is 
necessary to construct accurate shape representation of the real object in a computer 
system. A traditional method for this is to use conventional modeling software after 
precisely measuring the size or length of the real object by hand.". Thus, sections 5 and 
2 suggest using a computer input device, such as a 6 DOF tracker, to measure the real 
object in order to have accurate shape representation of the real object's constraining 
shape. 

Argument 2: 

On page 14 second paragraph applicant argues the determining step of claim 10: 
(2) determining an input of a constraining shape or an operation of the virtual object. 
Applicant argues "Applicants have found nothing in those passages that would teach or 
suggest "determining an input of a constraining shape or an operation of the virtual 
object," as recited in Claim 10.", however, the shape of the real objects are known in 
advance which means an input of a constraining shape corresponding to the real object 
was input and a determination of the constraining shape is made during the operation of 
the computer program. 

Argument 3 

In the paragraph spanning pages 14 and 15 as well as the following paragraph 
on page 1 5 applicant argues the obtaining step of claim 1 0: (3) obtaining a 
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constraining shape by using a shape generated based on the obtained three- 
dimensional position information in case of the input of the constraining shape. 
Applicant argues "Applicants strongly disagree. Kitamura's discussion of a method of 
precisely measuring the size or length of the real object by hand is completely 
contradictory to a suggestion of using a computer input device to measure the real 
object." and "Applicants respectfully disagree. Kitamura makes clear that the shapes of 
the real objects are assumed to be known in advance. Thus, there is no suggestion 
whatsoever of using the 6 DOF tracker to input the 3D coordinates of the constraining 
shapes.", however, to one of ordinary skill in the art the position and orientation of the 
real objects that will be interacted with the 6 DOF tracker would be efficiently known in 
advance by using the 6 DOF tracker to measure the real object by the usermoving the 6 
DOF tracker "by hand". 

These three arguments concerning the 35 USC 103 rejection have been fully 
considered but they are not persuasive because as discussed in applicants specification 
at paragraphs [0028], [0041], [0050], [0068], and [0078] alternative means for stylus 
1060 for inputting the 3D coordinates of the constraining shape is possible, thus, the 
claimed "inputting means for inputting three-dimensional position information of a 
plurality of positions inputted by moving an operating unit in the real space by a user, 
the operating unit being capable of measuring the position and orientation in the real 
space" of claim 1 ; "an inputting step of inputting three dimensional position information 
of a plurality of positions inputted by moving an operating unit in the real space by a 
user, the operating unit being capable of measuring the position and orientation in the 
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real space" of claim 5; "obtaining three-dimensional position information of a plurality of 
positions designated by an operating unit moved by a user in the real space, the 
operating unit being capable of measuring the position and orientation" of claim 10; and 
"an inputting unit configured to input three-dimensional position information of a 
plurality of positions inputted by moving an operating unit in the real space by a user, 
the operating unit being capable of measuring the position and orientation in the real 
space" of claim 1 6 may be means other than stylus 1 060 which alternative means is 
taught by 6 DOF tracker device of Kitamura. Applicant previously amended the setting 
means of claim 1 , the setting step of claim 5, the obtaining a constraining step of claim 
10, and the setting unit of claim 16 to claim "constraining shape by using a shape 
generated based on the inputted (or obtained) three-dimensional information". The 6 
DOF tracker device of Kitamura suggests using the 6 DOF tracker device to input 3D 
coordinates of the constraining shape since a means to input the constraining shape is 
inherently present and because the real object with inputted constrained properties is 
moved by the user using the 6 DOF tracker device which will allow for less input devices 
being used in Kitamura's system. Additionally section 2 at lines 7-14 states "To bring an 
object that already exist in the real world into a computer-generated virtual world, it is 
necessary to construct accurate shape representation of the real object in a computer 
system. A traditional method for this is to use conventional modeling software after 
precisely measuring the size or length of the real object by hand." which suggests using 
a computer input device to measure the real object in order to have accurate shape 
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representation of the real object's constraining shape. The previous 103 rejection is 
maintained and reproduced below with modifications reflecting claim amendments. 

Furthermore applicant appears to be arguing there is a claimed order to the 
claimed steps and functions, however, the claims do not claim any order for the claimed 
steps. Although the claims are interpreted in light of the specification, limitations from 
the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1 181 , 26 
USPQ2d 1057 (Fed. Cir. 1993). Thus, using Kitamura's 6DOF to input the position and 
orientation of the real objects to have the shapes "known in advance" before interacting 
with the virtual objects meets the steps of claims 5 and 10 and meets the functions of 
claims 1 and 16. See Interactive Gift Express, Inc. v. Compuserve Inc., 256 F.3d 1323, 
1342-43 (Fed. Cir. 2001). See also more recent decisions making reference to 
Interactive: 2007/06/29 06-1 584.pdf DCT Frazier v. Lavne Christensen Company N; 
2007/11/19 07-1 1 50.pdf PTQ In Re Chatani N; and Baldwin Graphic Systems Inc. v. 
Siebert Inc., 85 USPQ2d 1503, 1508 (Fed. Cir. 2008), 2008/1/15 07-1262.pdf DCT 
Baldwin Graphic Systems v. Siebert P. 

Also note the newly cited Fa article teaches: (2) determining an input of a 
constraining shape or an operation of the virtual object; and (3) obtaining a constraining 
shape by using a shape generated based on the obtained three-dimensional position 
information in case of the input of the constraining shape. 
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Claim Rejections - 35 USC §112 

4. The following is a quotation of the first paragraph of 35 U.S. C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

5. Claims 1-3 and 5-18 are rejected under 35 U.S.C. 112, first paragraph, as failing 
to comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. The 01/07/2008 amendment to independent 
claims 1, 5, and 16 "input three-dimensional position information of a plurality of 
positions inputted by moving an operating unit in the real space by a user" and the 
01/30/2007 amendment to claim 10 "obtaining three-dimensional position information of 
a plurality of positions designated by an operating unit moved by a user in the real 
space" is not conveyed by the originally field specification because applicants 
specification in paragraphs 30, 31 , 32, 44, 69, 82, 83, 84, and 85 discuss operation unit 
1090 as controlling the virtual object and in paragraphs 28, 30, 42, and 50 discuss 
stylus 1060 as setting the position and orientation of a plurality of positions in real 
space. The specification does not convey the operating unit 1090 is moved to measure 
the three-dimensional position information used "for setting a constraining shape by 
using a shape generated based on the inputted three-dimensional position information 
in case of the input of the constraining shape" (claims 1, 5, and 19) and used for 
"obtaining a constraining shape by using a shape generated based on the obtained 
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three-dimensional position information" (claim 10). Thus, applicants originally filed 
specification does not convey that applicant had possession of the amendments to 
claims 1,5, 10, and 16. 

Claim Rejections - 35 USC §103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

8. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 



Application/Control Number: 1 0/61 5,941 Page 1 1 

Art Unit: 2628 

under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

9. Claims 1-3 and 5-18 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yoshifumi Kitamura and Fumio Kishino, Consolidated Manipulation of Virtual and 
Real Objects, September 1997, Proceedings of the ACM symposium on Virtual reality 
software and technology, pages 133-138. Kitamura teaches an augmented reality 
system that uses object constraints to control the visual interaction between the virtual 
objects and the real objects. 

A detailed analysis of the claims follows. 

Claim 10: 

Kitamura teaches an information processing method for changing the position 
and orientation of a virtual object in mixed reality space obtained by combining a real 
space and a virtual space, said method comprising the steps of: 

obtaining three dimensional position information of a plurality of positions 
designated by an operating unit moved by a user in the real space (In view of page 135 
first paragraph under the heading of 5 Consolidated Manipulation Environment of 
Kitamura the same 6 DOF tracker device is used to control the position of a virtual world 
object(s) and to control the position of a constraining real world object(s) which is similar 
to applicants system where stylus 1060 is used to control the virtual world object(s) and 
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to control the location of the constraining real world object(s), see applicants paragraphs 
[0028], [0030], and [0041]. The 6 DOF has a stylus the user moves which meets the 
"operating unit moved by a user in the real space" claim limitation.), the operating unit 
being capable of measuring the position and orientation in the real space {The 6 DOF 
measures position and orientation in real space of the sylus on the 6 DOF.); 

determining an input of a constraining shape or an operation of the virtual 
object (The system determines if a shape having constraints is being input for the real 
object such as a toy block (section 4.1) or surface (section 5.4) or such as moving the 
real object which has a constraining shape and the system determines if the user is 
moving the virtual object); 

Kitamura does not fully teach: obtaining a constraining shape by using a shape 
generated based on the obtained three-dimensional position information in the case of 
the input of the constraining shape (The first paragraph in section 5 on page 135 
describes the user using a 6 DOF tracker device to manipulate the virtual world objects 
and to position the constraining real world objects. The shape of the real world object(s) 
are used to constrain the movement of the virtual world object(s) by giving the real world 
object(s) a shape that the virtual world object(s) interacts with in a constrained manner. 
A step of inputting the shape of the real world objects constraining shape is inherently 
present). Kitamura's use of the 6 DOF tracker device to control the position of the real 
world object suggests using the same 6 DOF tracker device to input the constraining 
shape of the real world object since this would require less input devices for the user to 
use and learn how to use. Additionally section 2 at lines 7-14 states "To bring an object 
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that already exist in the real world into a computer-generated virtual world, it is 
necessary to construct accurate shape representation of the real object in a computer 
system. A traditional method for this is to use conventional modeling software after 
precisely measuring the size or length of the real object by hand." which suggests using 
a computer input device to measure the real object in order to have accurate shape 
representation of the real object's constraining shape. Thus, it would have been 
obvious to one of ordinary skill in the art at the time of applicants invention to input with 
the 6 DOF tracker device the 3D coordinates of the constraining shape because the toy 
block (section 4.1 ) or surface (section 5.4) need to have their respective constraining 
coordinates input in order for the virtual object to properly interact with the real objects 
since this will require less input devices for the user to use and learn how to use and 
because section 2 at lines 7-14 suggests using a computer input device to measure the 
real object in order to have accurate shape representation of the real object's 
constraining shape. To one of ordinary skill in the art the position and orientation of the 
real objects that will be interacted with the 6 DOF tracker would be efficiently known in 
advance by using the 6 DOF tracker to measure the real object by the user moving the 
6 DOF tracker "by hand". KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 
2007), U.S. Supreme Court No. 04-1350 Decided April 30, 2007, 127 SCt 1727, 167 
LEd2d 705.; 

Kitamura further teaches changing the position and orientation of the virtual 
object according to instructions from the user, based on the obtained constraining 
shape as constraint condition in case of the operation of the virtual object (The first 
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paragraph in section 5 on page 135 describes the user using a 6 DOF tracker device to 
manipulate the virtual world object(s). The shape of the real world object(s) are used to 
constrain the movement of the virtual world object(s) by giving the real world object(s) a 
shape that the virtual world object(s) interacts with in a constrained manner. Sections 
5.1 to 5.4 discusses manipulation of the virtual world object based upon constraint 
conditions based on the shape of the real world object in response to the user using the 
6 DOF tracker device. The constraining shape generated from the three-dimensional 
position information constrains the interaction of the virtual world object with the real 
world object, see sections 2, 4 to 5.4. The introduction on page 133 second full 
paragraph discusses augmented reality which synthesizes a virtual object with a real 
object. Section 5. 1 discusses after movement is detected by the 6DOF manipulator the 
virtual object is moved according to the constraints.); and 

combining an image of the virtual object generated according to the changed 
position and orientation, and the real image (The introduction on page 133 second full 
paragraph discusses augmented reality which synthesizes a virtual world object with a 
real world object. Section 5. 1 discusses after movement is detected by the 6DOF 
manipulator the virtual world object is moved according to the constraints of the real 
world object. Sections 2, 4, and 5 discusses mixing the virtual and real world images of 
virtual and real world objects.). 

Claim 11: 

Kitamura teaches an information processing method according to Claim 10, 
further comprising the step of combining a virtual image indicating the constraining 
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shape with the real image (The introduction on page 133 second full paragraph 
discusses augmented reality which synthesizes a virtual object with a real object. 
Section 5. 1 discusses after movement is detected by the 6DOF manipulator the virtual 
object is moved according to the constraints. Sections 2, 4, and 5 discusses mixing the 
virtual and real world images of virtual and real world objects.). 
Claim 12: 

Kitamura teaches an information processing method according to Claim 10, 
wherein the constraining shape is a plane. (On page 136 in the text above figure 2 
discusses determining a plane and using the plane to constrain movement of the virtual 
object is discussed with regards to figure 2.). 

Claim 13: 

Kitamura teaches an information processing method according to Claim 10, 
wherein said changing step changing the position and orientation of the virtual object is 
carried out by changing the position and orientation of the operating unit (The 6 DOF 
tracker device is an operating unit. The user using the 6 DOF tracker device 
manipulates the virtual objects by changing the position and orientation of the 6 DOF 
tracker device.). 

Claim 14: 

Kitamura teaches a computer program product comprising a computer readable 
medium storing computer program code for performing the information processing 
method according to Claim 10 wherein the information processing method is executed 
by a computer device (This article is directed to computers that generate the augmented 
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reality scene since it was published by ACM for a symposium on virtual reality software 
and technology and since at page 133 in the last sentence in the second paragraph of 
section 1 "software/hardware" is discussed. Software causing a computer to perform 
Kitamura's augmented reality is inherently stored in a computer readable recording 
medium.). 

Claim 15: 

Kitamura teaches a computer-readable recording medium, storing the computer 
program according to Claim 14 (Software causing a computer to perform Kitamura's 
augmented reality is inherently stored in a computer readable recording medium.). 

Claim 16: 

The functions of Kitamura corresponds to the claimed units because the software 
causes the computer to become a unit that performs a process. As seen below 
Kitamura performs the claimed processes. 

Kitamura teaches an information processing device for aiding control operations 
relating to controlling the position and orientation of a virtual object, said device 
comprising: 

an image capturing unit configured to capture a real image in real space {The 
measurements by the user in three dimensional real space, the device for obtaining the 
3-D shape by using a range image, and the device for obtaining the 3-D shape by using 
multiple cameras captures a real image in real space. The three paragraphs found in 
section 2 on page 134 of Kitamura teaches the user measuring the real objects, a 
device measuring the real objects with range in response to the user, or a device 
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measuring the real objects with multiple cameras range in response to the user. The 
claim does not limit the form of the real image, thus, Kitamura teaches this limitation. 
Note applicants paragraph [0024] does not restrict the image capturing unit to a 
particular type); 

a virtual image generation unit configured to generate a virtual image of a virtual 
object according to the position and orientation of said image capturing unit {Kitamura 
discusses in section 2 in the first paragraph using conventional modeling software after 
precisely measuring the size or length of the real object by hand or by the devices which 
is an generation unit capable of generating three-dimensional positional information. 
Sections 2, 4 and 5 discuss generating a virtual image(s) of a virtual object(s).) ; 

a superimposing unit configured to superimpose the generated virtual image with 
the captured real image (Sections 2, 4, and 5 discusses mixing the virtual and real world 
images of virtual and real world objects.); 

a determination unit configured to determine an input of a constraining shape or 
an operation of the virtual object (The system determines if a shape having constraints 
is being input for the real object such as a toy block (section 4. 1) or surface (section 5.4) 
or such as moving the real object which has a constraining shape and the system 
determines if the user is moving the virtual object); 

an inputting unit configured to input three-dimensional position information of a 
plurality of positions inputted by a moving a operating unit in the real space by a user (In 
view of page 135 first paragraph under the heading of 5 Consolidated Manipulation 
Environment of Kitamura the same 6 DOF tracker device is used to control the position 
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of a virtual world object(s) and to control the position of a constraining real world 
object(s) which is similar to applicants system where stylus 1060 is used to control the 
virtual world object(s) and to control the location of the constraining real world object(s), 
see applicants paragraphs [0028], [0030], and [0041]. The 6 DOF has a stylus the user 
moves which meets the "moving a real object limitation in the real space by a user" 
claim limitation.), the operating unit being capable of measuring the position and 
orientation in the real space (The 6 DOF measures position and orientation in real 
space of the sylus on the 6 DOF.); 

Kitamura does not fully teach: a setting unit configured to set a constraining 
shape bv using a shape generated based on the inputting three-dimensional position 
information in the case of the input of the constraining shape {The first paragraph in 
section 5 on page 135 describes the user using a 6 DOF tracker device to manipulate 
the virtual world objects and to position the constraining real world objects. The shape 
of the real world object(s) are used to constrain the movement of the virtual world 
object(s) by giving the real world object(s) a shape that the virtual world object(s) 
interacts with in a constrained manner. A step of inputting the shape of the real world 
objects constraining shape is inherently present). Kitamura's use of the 6 DOF tracker 
device to control the position of the real world object suggests using the same 6 DOF 
tracker device to input the constraining shape of the real world object since this would 
require less input devices for the user to use and learn how to use. Additionally section 
2 at lines 7-14 states "To bring an object that already exist in the real world into a 
computer-generated virtual world, it is necessary to construct accurate shape 



Application/Control Number: 10/615,941 Page 19 

Art Unit: 2628 

representation of the real object in a computer system. A traditional method for this is to 
use conventional modeling software after precisely measuring the size or length of the 
real object by hand." which suggests using a computer input device to measure the real 
object in order to have accurate shape representation of the real object's constraining 
shape. Thus, it would have been obvious to one of ordinary skill in the art at the time of 
applicants invention to input with the 6 DOF tracker device the 3D coordinates of the 
constraining shape because the toy block (section 4.1) or surface (section 5.4) need to 
have their respective constraining coordinates input in order for the virtual object to 
properly interact with the real objects since this will require less input devices for the 
user to use and learn how to use and because section 2 at lines 7-14 suggests using a 
computer input device to measure the real object in order to have accurate shape 
representation of the real object's constraining shape. To one of ordinary skill in the art 
the position and orientation of the real objects that will be interacted with the 6 DOF 
tracker would be efficiently known in advance by using the 6 DOF tracker to measure 
the real object by the user moving the 6 DOF tracker "by hand". KSR International Co. 
v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007), U.S. Supreme Court No. 04-1350 
Decided April 30, 2007, 127 SCt 1727, 167 LEd2d 705.; and 

Kitamura further teaches an operating unit configured to control the position and 
orientation of the virtual object based on the constraining shape in accordance with the 
a user's instruction in case of the operation of the virtual object. ( Sections 5.1 to 5.4 
discusses manipulation of the virtual world object based upon constraint conditions 
based on the shape of the real world object in response to the user using the 6 DOF 
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tracker device. The constraining shape generated from the three-dimensional position 
information constrains the interaction of the virtual world object with the real world 
object, see sections 2, 4 to 5.4. The introduction on page 133 second full paragraph 
discusses augmented reality which synthesizes a virtual object with a real object. 
Section 5. 1 discusses after movement is detected by the 6DOF manipulator the virtual 
object is moved according to the constraints.). 
Claim 17: 

Kitamura teaches an information processing device according to Claim 16, 
wherein the constraining shape is defined by polygons {Figure 2's Cubes are formed 
with polygons.) and the apexes of the polygons are at positions inputted by the user {As 
discussed in the a setting unit above it would have been obvious to use the 6 DOF 
tracker device to input the constraining shape of the real world object which would input 
the apexes of the constraining shape of the polygon.) or the constraining shape is a 
plane passing through the positions inputted by the user {A real sensed surface is a 
planar real world object whose position is inputted by the user in the setting unit. As 
discussed in the a setting unit above it would have been obvious to use the 6 DOF 
tracker device to input the constraining shape of the real world object which would input 
plane for a surface.). 

Claim 18: 

Kitamura teaches an information processing device according to Claim 16, 
wherein said operating unit performs at least one of the following operations in 
performing an operation controlling the position and orientation of the virtual object: 
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a translation operation for causing translational movement of the virtual object 
based on the constraining shape {Page 137 column 1 lines 1-3 teaches translation 
when the virtual object collides with the real surface.); or 

a rotation operation for rotating the virtual object on an axis which is a normal 
vector at a plane where the constraining shape and the virtual object come into contact 
{Page 137 column 1 lines 1-3 teaches rotation when the virtual object collides with the 
real surface.) {Page 137 column 1 lines 1-3 also teaches translation and rotation when 
the virtual object collides with the real surface). 

Claims 1-3: 

Means plus function claims 1-3 correspond to device claims 16-18 and the 
means of Kitamura, software and computer, are equivalent to applicant's means of 
software and computer. 

Claims 5-7: 

Step for claims 5-7 correspond to device claims 16-18 and the steps of Kitamura, 
software and computer, are equivalent to applicant's steps performed by software and 
computer. 

Claims 8 and 9: 

Claims 8 and 9 mirror claims 14 and 15 addressed above and they are rejected 
for the same reasons given above for claims 14 and 15. 
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Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

IBM Technical Disclosure Bulletin, September 1997, Vol. 40, No. 9, pages 79- 
82.This document discusses "Navigation constraints in a Virtual Reality environment are 
implemented by placing collision boundaries around an otherwise punctual, navigable 
view point. By adjusting these collision boundaries according to the view point's field of 
view, the designers can implement cheaply, useful navigation constraints that will 
prevent users to move their points of view too close to objects and loose sight of the 
environment's context." 

Terrence Fernando, Norman Murray, Kevin Tan, Prasad Wimalaratne; Software 
Architecture for a Constraint-based Virtual Environment; Year of Publication 1999; 
Virtual Reality Software and Technology, Proceedings of the ACM symposium on 
Virtual reality software and technology; pages 147 - 154. This document discusses 
constraint based modeling in section 2.2. 

Mingxian Fa, Terrence Fernando, Peter M. Dew; Interactive Constraint-based 
Solid Modeling using Allowable Motion; Year of Publication: 1993; ACM Symposium on 
Solid and Physical Modeling, Proceedings on the second ACM symposium on Solid 
modeling and applications; pages: 243 - 252. This document discusses in the first and 
second paragraphs of the introduction and in section 3.2 allowing the user to define 
constraints stored in Relationship Graph (RG) for use during manipulation of virtual 
objects. 
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1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jeffery A Brier whose telephone number is (571) 272- 
7656. The examiner can normally be reached on M-F from 7:30 to 4:00. If attempts to 
reach the examiner by telephone are unsuccessful, the examiner's supervisor, Michael 
Razavi, can be reached at (571 ) 272-7664. The fax phone Number for the organization 
where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



/Jeffery A. Brier/ 

Primary Examiner, Division 2628 



